Non-Parametric Kernel Learning with robust pairwise constraints

For existing kernel learning based semi-supervised clustering algorithms, it is generally difficult to scale well with large scale datasets and robust pairwise constraints. In this paper, we proposed a new Non-Parametric Kernel Learning framework (NPKL) to deal with these problems. We generalized the graph embedding framework into kernel learning, by reforming it as a semi-definitive programming (SDP) problem, smoothing and avoiding over-smoothing the functional Hilbert space with Laplacian regularization. We proposed two algorithms to solve this problem. One is a straightforward algorithm using semidefinite programming (SDP) to solve the original kernel learning problem, dented as TRAnsductive Graph Embedding Kernel learning (TRAGEK); the other is to relax the SDP problem and solve it with a constrained gradient descent algorithm. To accelerate the learning speed, we further divide the data into groups and used the sub-kernels of these groups to approximate the whole kernel matrix. This algorithm is denoted as Efficient Non-PArametric Kernel Learning (ENPAKL). The advantages of the proposed NPKL framework are 1) supervised information in the form of pairwise constraints can be easily incorporated; 2) it is robust to the number of pairwise constraints, i.e., the number of constraints does not affect the running time too much; 3) ENPAKL is efficient Changyou Chen Research School of Information Sciences and Engineering, The Australian National University, Canberra, Australia E-mail: [email protected] This work was done when I was at Fudan University, Shanghai, China Junping Zhang Shanghai Key Laboratory of Intelligent Information Processing and School of Computer Science, Fudan University, Shanghai, China Tel.: +123-45-678910 Fax: +123-45-678910 E-mail: [email protected] Xuefang He School of Software and Information Engineering, Beihai College of Beihang University E-mail: [email protected] Zhi-Hua Zhou National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China Tel.: +123-45-678910 Fax: +123-45-678910 E-mail: [email protected] 2 Changyou Chen et al. to some extent compared to some related kernel learning algorithms since it is a constraint gradient descent based algorithm. Experiments for clustering based on the learned kernels show that the proposed framework scales well with the size of datasets and the number of pairwise constraints. Further experiments for image segmentation indicate the potential advantages of the proposed algorithms over the traditional k-means and N -cut clustering algorithms for image segmentation in term of segmentation accuracy.